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Approxima,tions  for  Functionals  and  Optimal  Control  Problems 
on  Jiunp  Diffusion  Processes 


Abstract 

The  paper  treats  approximations  to  stochastic  differential 
equations  with  both  a diffusion  and  a jump  component,  and  to 
associated  functionals  and  partial-differential-integral  equations 
of  the  (degenerate  or  not)  elliptic  or  parabolic  type.  Approxima- 
tions for  the  optimal  control  problem  on  such  a model,  or  for  the 
associated  non-linear  partial-differential-integral  equation  is 
discussed.  The  techniques  are  purely  probabilistic  and  are 
extensions  of  those  in  [3] , which  dealt  with  the  diffusion  case. 


1.  Introduction 


r 


The  paper  treats  approximations  for  the  process  (1.1) 
tterms  to  be  defined  below) , and  associated  functionals,  partial 
differential  integral  equations,  and  for  certain  optimal  control 
problems  (where  f and  k can  depend  on  a control  parameter  u) . 


(1.1) 


X(t) 


X + [ f CX  (s)  )ds  + 

J 0 


a(X(s)  )dw(s) 

0 


+ 


■t 

0 


g (X  (s  ) ,a)Q (daxds) , 


t > 0, 


where  QC*f") 
process  Q(t) 


is  a Poisson  measure,  with  associated  Poisson 
rt  , 

aQ(daxds).  The  process  (1.1)  is  a widely 


used  model  for  situations  where  there  is  a juitp  and  a diffusion 
component  to  the  process  paths  [1] . 

Define  t = inf{t:  X(t)  ft  G) , where  G is  a given 
bounded  open  set,  assume  that  E^x  < «>,  and  define  the  functional 


a. 2) 


RCx) 


E 


X 


■T 

k(XCs)  )ds  + E^({)  (X{t)  ) . 
0 ^ 


Let  Q(‘)  have  jump  rate  c,  and  jump  distribution 
pC*).  Define  the  measures  tt  t*  ^ * ) /Tt  ( • ) ,f  (x,  • ) and  T(x,*) 
by  (for  Bor  el  A and  t > s) 


I 
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1T(A,[s,t])  = EQ(AX[s,t])  = TT(A)(t-S), 

A 

r(x,A)  =cr(x,A)  = ir(a:  g(x,a)  e A)  = cp(a:  g(x,a)  e A). 

^ into  A 

r(x,A)  is  the  jump  rate' at  t of  the  last  integral  in  (1.1), 
when  X(t)  = X.  Under  certain  smoothness  assumptions  on  R(*), 
it  satisfies 

(1.3)  ^V(x)  + I [V(x+a)  - V(x)jr(x,da)  + k(x)  = 0,  x e G, 

V(x)  = <J)  (x)  , X G, 


the  term  S/f  is 


, 2a  (x)  = a(x)a'  (x)  , 


the  differential  generator  of  the  diffusion  part  of  (1.1). 
We  use  the  following  conventions.  If  Q(*)  jumps  a at 
time  t,  then  in  (1.1),  g(X(t”),a)  is  the  increment  of  the 
integral  at  time  t.  Let  Y(*)  be  constant  on  [tj^,t2>. 


Then 


g (Y  (tj^)  ,a)  Q (daxds)  . The 


g(Y(s  ),a)Q(daxds)  = 

-1'  ^1 
integration  is  over  (t^,t2l  only. 

The  main  aim  of  the  paper  is  to  develop  computable 


approximations  to  various  functionals  of  (1.1),  such  as  (1.2), 
with  or  without  a control.  The  approximations  are  also 
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approximations  to  weak  solutions  of  equations  such  as  (1.3) 

(or  their  non-linear  counterparts  in  the  controlled  case) . 

As  in  [2],  we  exploit  the  close  relationship  between  (1.2) 
and  Cl. 3)  to  develop  approximations  to  both.  The  method  can 
be  used  to  approximate  a broad  variety  of  functionals  of 
Cl.l).  (See  Theorem  7.1.)  Functional  (1.2)  is  a very  special  case. 

The  technique  is  like  that  used  in  [2]  or  in  [3]  (we  use 
results  in  [3]  whenever  convenient)  for  functionals  of 
diffusions,  and  for  equations  such  as  (1.3)  without  the 
integral  term.  See,  also  [4] , where  a simpler  jump  problem 
is  treated.  By  using  suitable  finite  difference  approximations 
to  (1,3),  we  can  construct  a certain  Markov  chain.  Suitable 
interpolations  of  the  chain  converge  weakly  to  (1.1),  and  the 
chain  can  be  used  to  compute  approximations  to  (1.2)  or  (1.3) 

(or  to  other  suitable  path  functionals) . This  is  true  whether 
or  not  (1.3)  has  a smooth  solution.  Under  conditions  to  be 
given  below,  the  approximations  converge  to  the  correct  values 
as  the  difference  intervals  go  to  zero.  Actually,  the 
difference  method  is  only  one  of  many  that  can  be  used  - this 
will  be  clear  in  the  sequel.  The  f.d.  method  is  used  only  to 


get  a "consistent"  sequence  of  approximations  to  X(*).  Any 
other  technique  for  doing  this  can  also  be  used.  If  f and  k 
depend  on  a control,  then  the  technique  is  useful  for  the 
approximation  of  optimal  controls.  Then,  the  approximating 


chain  becomes  a controlled  chain.  See  [2]  or  (3]  for  an 


introduction  to  the  general  technique  and  background.  For 


simplicity,  only  the  homogeneous  case  will  be  treated.  The 
non-homogeneous  case  and  parabolic  versions  of  (1.3)  are 
treated  similarly  to  the  case  here  (see  [3],  Chap.  7).  Also, 
reflecting  boundaries  can  be  added. 

Assumptions,  r and  r'  are  given  integers 

CAl.l)  f(-),  a(-),  k(-),  <}){•)  are  bounded  con- 
tinuous R^,  r X r matrix,  R and  R valued  functions,  resp. , 
on  R ; g { • , • ) is  a bounded  measurable  R valued  function 
on  R X R , and  continuous  in  its  first  argument  for  each 
value  of  the  second. 


r * 

(Al.2)  Q(t),  t e [0,“)  is  a R valued  Poisson 

process  with  jump  rate  c,  and  jump  distribution  p(*).  Let 
Q(daxds)  denote  the  associated  Poisson  measure  [1]  or  [5], 
Chapter  6 . Let  w ( • ) be  a standard  R^  valued  Wiener  process 
independent  of  Q(*). 

CAl . 3 ) The  process  (1.1)  has  a unique  non-anticipative 
solution,  for  each  non-ant icipative  (with  respect  to  w(*), 
Q(*f*l)  initial  condition  x,  with  and  without  the  jump  term. 

By  uniqueness,  we  mean  that  the  solutions  - for  any  w(*), 

Q(*»*l  satisfying  (Al.2)  - all  induce  the  same  measure  on 
D [0,")  (see  [ 3 ] or  [ 6 ] for  a discussion  of  D[0,“),  the  space 
of  right  continuous  functions  with  left  hand  limits) . If 
c < oo,  then  (A1.3)  holds  if  it  holds  with  g = 0. 
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(Al.4)  Cto  be  dropped  in  Section  8;  see  below  (2.1) 
for  Qj^Cx)) 

2 

supCh  /Qj^(x))  0 as  h 0. 

CAl.5)  E T < ” for  values  x of  interest. 

X 

Section  2 introduces  the  finite  difference  approximation 
and  relates  it  to  a Markov  chain.  The  finite  difference  solu- 
tion is  a functional  of  the  chain.  In  Sections  2 and  3,  a 
continuous  time  interpolation  of  the  chain  is  introduced.  The 
interpolated  process  has  a diffusion,  drift  and  jump  component, 
and  Section  4 discusses  the  properties  of  the  weak  limit  of  the 
jump  component,  and  also  shows  that  the  weak  limit  (as  the 


finite  difference  interval  goes  to  0)  of  the  interpolation 
is  the  process  (1.1).  Section  5 discusses  an  alternative 
representation  of  the  jump  component,  which  is  particularly 
useful  in  approximations  to  optimal  control  problems  for  con- 
trolled versions  of  (1.1),  or  for  certain  non-linear  forms  of 
(1.3).  The  section  also  contains  an  alternative  Markov  chain 
with  which  the  computation  is  a little  simpler.  The  case 
c = 00  is  treated  in  Section  6,  and  Section  7 deals  with  the 
convergence  of  functionals  of  the  chain  (or  of  the  finite 
difference  approximation) to  functionals  of  (1.1).  In  Section  8, 
we  develop  an  alternative  continuous  time  interpolation,  which 
is  a Markov  jump  process  and  which  also  converges  weakly  to  (1.1). 


1 
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As  with  the  earlier 

interpolation,  the  finite  difference  solution  - or  chain  func- 
tional - is  also  a functional  of  the  interpolated  process,  and 
this  functional  converges  to  the  correct  functional  of  (1.1), 
as  the  finite  difference  interval  goes  to  0.  The  limit  func- 
tional is  the  desired  solution.  Section  9 contains  some 
remarks  on  the  optimal  control  problem. 

2.  The  Discrete  Approximation 

Until  Section  6,  we  assume  c < ».  The  operator  in 
Cl. 3)  will  be  discretized  using  the  f.d.  (finite  difference) 
approximations  of  [2]  or  [3],  Chapter  6.2,  for  V (*),V  (•)• 

^ Xi  X.x. 

Let  h = finite  difference  interval,  and  e^  = unit  vector  in 
i^^  coordinate  direction.  Let  = finite  difference  grid  on 
R^,  and  define  = R^  P)  G.  A convenient  way  to  discretize 

the  integral  in  (1.3)  is  as  follows.  For  each  set  of  integer s^^ 
jf,...,jr  and  "finite  difference  box",  b^ ( j^, . . . , j^)  = 

(j.h-h,j.h],  let  j'h,...,j'h  denote  the  grid  point  in  the 

i=l  1 1 ^ 

closure  of  the  box  which  is  closest  to  the  origin. 

^For  simplicity,  we  let  h be  independent  of  the  direction.  More 
general  schemes  are  possible. 

^^The  can  be  positive,  negative  or  zero. 
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(2.2)  [Q.  (x)  + ch^]V^(x)  = I Q.  (x)p*’(x,x±e.h)V^(x±e.h) 


i/D 

+ 


+ Q.  (x)p^(x,x±e.h+e  .h)  V^(x±e.h+e  .h) 

ill  1 j 1 ] 


If] 


+ ch 


I r^(x,j^,...,j^)V^(x+E  e^jjh)  + h^k(x), 


3 2_'  ’ ’ ’ ' ^ j- 


V (x)  = 0 (x)  , X ^ Gj^. 


x e G^, 


We  can  work  with  (2.2),  or  with  various  approximations 
to  (2.2).  Equation  (2.2)  will  be  put  into  a slightly  more 
convenient  form.  Define  At^(x)  = h^/ [Q^ (x) +ch^] , 


(2. 3a)  P^Cx)  = 1 - {Q^ix)  / iQ^(x)  + ch^))  = cAt^(x) 


(2.3b)  P^(x)  = 1 - exp  - cAt^(x). 


Rewrite  (2.2)  as  (for  x e G,  ) 

h 


(2.4)  V^(x)  = (1-P^(x))[^  P^(x,y)V^(x,y)  J 


+ P^(x) [ I r^(x,j  ,...,j  )v^(x  + I e.j!h] 


+ At  (x) k (x) , 


where  P is  P or  P.  If  P is  used,  then  (2.4)  = (2.2).  The 

A 

P and  P are  first  order  approximations  (in  At)  to  each 
other;  the  limits  do  not  depend  on  which  is  used.  For 
notational  uofiniteness  we  use  (2.3b)  (which  seems  to  give 

better  numerical  results  also) . 

Suppose  that  the  T^,P^,p^  are  defined  on  all  grid  points  x 
(or  grid  point  pairs  for  p“)  on  R^.  Then  the  coefficients 
in  (2.4)  have  the  following  interpretation.  The  p^(x,y) 
are  in  (0,1]  and  sum  to  unity  over  y,  for  each  x,  and  so 
do  the  r^(x,  j^, . . . , j^)  (summed  over  j , jj.)  • Also, 

P^(x)  e [0,1].  Thus,  the  system  p^(x,y),  P^(x)}  has  the  following 


interpretation  in  terms  of  a Markov  chain  • •'  on  the  state 
,r  ,h  .,h 


;..h- 

n 


space  R^.  Let 


-h 

r,  = X. 

n 


Then  w.p.  (1-P  (x) ) , use  the  transition 

hr  h 

probabilities  {p”(x,y),  y c Rv, ' » and  w.p.  P (x)  , use 


(x,jj^,...,j^)  — P(C 


n+1 


= X 


second  instance,  we  say  that  the  process  has  jumped  at  n+1  - even 
if  all  j|  are  zero. 

Let  N,  = min{n:  if  G.  }.  Assume,  only  for  the  moment, 

that 


E N.  < ®, 
X h 


Then  (2.4)  has  a unique  solution  which  is 


N^-1 


C2.5)  V^(x)  = I k(cJj)At[j  + E^<,(cjj  ), 


n=0 


xo- 


where  At^  = At^(C^). 

n n 

We  will  next  develop  a convenient  representation  for 
the  process,  and  then  prove  various  properties  of  the 

weak  limits,  ultimately  showing  that  (when  suitably  inter- 
polated), the  chains  converge  weakly  to  the  solution  to  (1.1), 
and  V^(x)  ->  R(x),  as  h ->  0.  Similarly,  many  other  functionals 
of  X(*)  can  be  approximated  by  functionals  of  (Cn}r  with 
convergence  as  h -*■  0 (Theorem  7.1). 

3.  Properties  of  the  Process 

We  have  (see  [2]  or  [3] , Chap.  6,  for  the  calculation  in 
the  "no  jximp"  case)  , 

(3.1)  ^n  ~ jump  at  n+1]  = f(x)At^(x) 

(3.2) ”^  Covar  I ~ jump  at  n + 1]  = Jj^(x)At^(x) 

= 2a(x)At^(x)  + At^(x)[hf(x)  - f (x)  f ' (x)  At^  (x)  ] 

Define  3^^  = ~ ^n  ~ there  is  a jump 

at  n + 1,  we  alter  the  definition  and  assume  (to  calculate  3 
only)  that  evolves  from  as  though  there  were  no 


f (x)  is  the  diagonal  matrix  with 


ith 


entry 


fi(x) 


x:  c 
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jump.  Henceforth,  let  us  fix  the  initial  condition  x = 


Write 


(3.3) 


X + fJJ  + bJJ  + jjj  - eJJ, 

n n n n 


K h V,  K 


n-1  . 
r h 

I e., 

i=0  ^ 


where  = 1 if  there  is  a jump  at  i + 1,  and  is  zero  other- 


wise. 


h h 

Define  the  piecewise  constant  interpolations  c"(*)# 


etc.  exactly  as  done  in  [2]  or  [3],  Chapter  6.  E.g.,  define 

tjj  = I At^,  and  set  F^(t)  = fJ^  on  , ).  Define 

I*  Q X n n lit  X 

Vk  Vi  Vi  V» 

m^  and  6m^  = m?  - i > 2,  and  m^  = 6m^,  to  be  the 

i^^  jump  time  and  inter jump  interval  resp. , for 

and  6t^  “ ^i  ~ ^i-1'  i > 2,  and  x!^  = 6x^  be  the  i^^  jump 
time  and  interjump  interval,  resp.  for  Define  R"*"  = [0,°°) 


Theorem  3.1.  Under  (Al.l),  (Al.2),  and  (Al.4),  the 
sequence  {$^(-),  6tJ,  i = 1,...}  = { ( • ) ,F^ ( • ) ,B^ ( • ) , J^( • ) , 
E^C*),6x^,  i = 1,2,...}  is  tight*"  on  D^^[0,«>)  x (R^)°°.  Also 
E^(’)  converges  to  the  zero  process  as  h -»•  0. 


We  say  that  the  sequence  of  processes  is  tight  if  the  corre- 
sponding sequence  of  measures  is  tight. 


follow  from 

lim  E max  |E^(t) |^  = 0,  each  T < “. 
h->0  t<T 

Vi 

Tightness  of  {F  C*)fB  (•))  is  proved  as  in  [3  ],  Theorem 
6.3.1.  The  average  number  of  jumps  of  J (•)  on  [0,t]  is 


M at  i)\th<t)  ' i 

! 

E y I . (l-exp-cAt^  ,)  < ct. 
i {t^<t} 

Boundedness  of  g(*,*^  and  (3.4)  imply  tightness  of 

since  (3.4)  implies  that  the  average  number  of  jumps  on  any  i 

finite  interval  is  bounded  independently  of  h.  Tightness  of  \ 

{^^(•)}  follows  from  the  above  tightness  results.  Let  9(h)  j 

denote  any  function  such  that  6(h)  ->^0  as  h 0.  Finally,  we 
can  show  that  (see,  e.g.,  the  calculations  in  proof  of  Theorem  4.1) 


h •“CTT 

P{6Xj^  > T}  < e + 0(h),  each  T < «>, 


h 


which  implies  tightness  of  {6t^,  i > D for  any  sequence 


h -»■  0. 


Q • £)  • D • 
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Theorem  3.2.  Assume  (Al.l),  (Al.2)  and  (Al.4).  Let 
h index  a weakly  convergent  subsequence  of  = 

i > 1}.  Denote  the  limit  process  by  $ = 
i > 1}.  The  process  $(•)  can  be  chosen  to 
have  right  continuous  paths,  also 


^ (t)  = X + F(t)  + B(t)  + J(t)  , 


where  F(t)  = 


f(5(s))ds,  and  there  is  a standard  Wiener 


0 


process  Wt»),  with  respect  to  which  all  the  other  terms  of 
$ are  non-anticipative'^and 


BCt)  = I a (C  (s)  )dW(s)  , t < »,  w.p.l. 
0 


Note.  All  the  jumps  of  JC*)  occur  at  the  {x.}, 

1 ^ 

6Xj,  i > 1,  but  there  can  be  a jump  of  zero  magnitude 

at  any  x^^.  For  later  use  (and  until  mentioned  otherwise)  , 
we  say  that  J C* ) or  ?(•)  jump  at  each  x^^  - even  if  the 
jump  is  of  zero  magnitude.  Thus,  J(*)  jumps  at  t,  ^ ~ ^i 

for  some  i.  Using  Skorokhod  imbedding,  C^(x^)  - C^(x^~)  -*■ 

C (x^)  - C(x^)  H 65  Cx^)  w.p.l,  as  h -►  0,  each  i.  Also,  to 
get  W(*)  we  may  have  to  augment  the  probability  space  by 
adding  an  independent  Wiener  process. 

^By  non-anticipative  x^  we  mean  that  the  process  with 

values  I^(t)  = is  non-anticipative. 
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Proof . The  continuity,  martingale,  and  representation 
properties  of  B(*)  are  proved  as  in  [3  ],  Theorems  6.3.1 
and  6.3.2.  In  the  proof  of  the  latter  theorem,  we  replace 
and  C (s^)  by  CC^Cs^)  , J^(s^),  H t)  and  (C(s^),  J(s^), 
n t)  , resp. , and  use  only  times  s^^  at  which  5(*),J(*) 
are  continuous  w.p.l.  The  rest  of  the  details  are  omitted. 

Note  that; 


(3.5) 


V^tx)  = kCC^(s))ds  + E^())(C^(Pj^)), 


h 1*1 

where  = min{t;  5”(t)  ^ G)  = t^  . The  representation  (3.5) 

h ^ h 

for  v“C*)  in  terms  of  a functional  of  C (•)  is  critical 

for  our  method,  because  we  show  that  the  representation  (3.5) 

actually  converges  to  R(x) . This  also  holds  for  the  inter- 


aolation  of  Section  8, 


Properties  of  J(*) 


In  this  and  in  the  next  section,  we  will  give  two 
methods  for  showing  that  there  is  a Poisson  measure  Q(*,*)  with 
the  same  properties  as  Q(«,»)  (and  a corresponding  Poisson 
process  S’(*))  such  that  W(*)  and  Q(*)  are  independent 


J(t)  = g(C(s  ) ,a) Q (daxds)  , t < «,  w.p.l. 

0 J 


Let  denote  the  a-algebra  determined  by 

j < n , and  O n,  all  j}.  Let  ^^(t)  be  the 

h 

a-algebra  determined  by  (5  (s)  , s < t,  and  f'l  t,  all  j}. 

Let  ^(t)  , S/^  and  S^.  , resp.  , be  the  a-algebras  determined 
by  {$(s) , s < t,  and  H t,  all  j },  (C  (s),  s < r^,  and 
^ {^>(s)  , s < Tj , and  Xj^»  k < j },  resp.  Let 

NCt,t+Al  and  N (t,t+A]  , resp.,  denote  the  number  of  jumps  of 
CC*)  and  resp.,  in  the  interval  (t,t+A]  , A > 0. 


Theorem  4.1.  Assume  (Al.l)»  (Al.2),  and  (Al,4)  and  consider 
the  converaent  subsequence  of  Theorem  3.2.  Then 


(4.1) 


^^(t)  = 0}  = exp  - cA 

P^^t)  = 1}  = CA  + o(A), 


where  the  o(A)  are  uniform  in  o),t.  The  {6x.},  6x.  = x.  - x. 
are  independent,  and  each  is  exponentially  distributed  with  mean 


Also, 

P^{6C(-ri)  = 5Ct^)  - 

CCt^) 

e A}  = 

■^i 

) 

e A) 

p{a:  g(C(T^),a) 

= r{^(x^),A) 

1,  for  each  Borel  set  A e 

R^. 
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Proof.  Part  1.  We  first  prove  the  analog  of  (4.1)  for 

^^(•).  Fix  h,  and  define  ^i2 ' ^i3 ' ^i4 

dicators  of  the  sets  ((t,s]  = empty  if  s < t) . 


u - <«''<•> 

jumps 

in 

u = 

jumps 

at 

tj, 

X3  = <5^•) 

jumps 

in 

[tJ^3^,t+A]  } 

i4  - ^ 

(t, t-i-A]  } 

• 

Note  that  is  measurable.  Now 


P . {N^Ct,t+A]  > 1}  < E , 
^^Ct)  ■ ~ ^""(t) 


i4 


f 


(4.3) 


Eh  ^ 

^^(t)  i 


i4"  ^h^i2 
^i-1 


^ h ^ I. . (l-exp-cAtJ_^) 
^ (t)  i 


cA  + c max  At  (y) 

y 


= cA  + 0 (h) , 


by  CAl.4) . Also 


P . {N^Ct,t+A]  > 2} 
^^Ct) 


I 

i 


^i4(l-"il) ^12^13 


(4.4)  < E . I I .(l-I  ,)I.  (cA+e(h)) 

^ (t)  i ^ 

< + 0 (h)  , 
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C4.5)  P V,  {N^Ct,t+A]  = 1}  = E ^ I 


^^(t) 


^“(t)  i 


Using  1 - cA  + 9 Ch)  < E ^ (which  follows  from 

C4.4)  ) 


t * 9(h), 


we  get  that  the  r.h.s.  of  (4.5)  is 


C4.6) 


cA  + OtA  ) + e (h)  . 


By  C4.3)  and  the  weak  convergence. 


C4.7) 


P{^C*)  jumps  at  t}  = 0,  each  t < 


Thus,  for  each  t,  ?(•)  is  continuous  at  t,  w.p.l.  Define 
I^(t,t+A;  xC*))  to  be  the  function  on  D^[0,»)  x (R^)°°  = S 
which  is  1 if  there  are  i jumps  of  x(*)  on  (t,t+A],  and 
is  zero  otherwise.  The  generic  element  of  S is  x(*)f 
{6a^,  i = 1,2,...}  where  is  the  i inter jump  interval 

for  a zero  or  a non-zero  jump  value.  Then,  by  (4.7),  the  function 
is  continuous  w.p.l  on  S,  with  respect  to  the  {5t^} 


measure. 


I 
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Let  mC*)  denote  a bounded  continuous  real  valued 
function  on  the  appropriate  space,  q and  t,,...,t  < t 

arbitrary,  and  note,  by  (4.3)  - (4.7)  and  the  weak  convergence. 


(4.8) 


Em(C^Ctj)  , Tj  lit,  j < q)I^(t,t+A;C^(*)  ) > 

tmC?(tj),  Tj  n t,  j < q)I^(t,t+A;C(-)  ) , 


as  h 0. 

The  in  the  l.h.s.  of  (4.8),  can  be  replaced  by 

2 9 

(l-cA+0(A  ) + 0(h))  when  i = 0,  and  by  cA  + 0 (A  ) + 6(h) 
when  i = 1. 

The  same  replacements  (without  the  9(h))  can  be  made 
in  the  r.h.s.  of  (4.8).  Consequently,  we  have  the  relation 
(i  = 0 or  1) 

Em(C(tj),Tj  n t,  j < q)I^(t,t+A;C(-)) 

= Em(C(tj),  Tj  n t,  j < q)P  {N(t,t+A]  = i} 

= Em(^(tj),  Tj  n t,  j < q)  [ (1-cA+o  (A) ) or  (cA+o(A))], 

which,  together  with  the  arbitrariness  of  m(*)»  q»  and  t^, 
i < q,  imply  that  (4.1)  holds  with  r.h.  sides,  1 - cA  + o(A), 
cA  + o(A),  o(A),  resp.  From  this,  it  is  easy  to  show  that  (4.1) 
holds  as  stated.  The  sentence  following  (4.1)  follows  from  (4.1). 
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11 

1. 

i; 


' • 


I. 

1 


Pa,rt  2.  Fix  i and  let  A be  a closed  Borel  set  in 
such  that  it  is  the  closure  of  its  interior  and 

Ep{a:  gCCCxT),a)  e 9A} 

= P{6?Ct^)  e 3A}  = 0. 

(I.e.,  if  q has  the  distribution  p(*)  and  is  independent 
of  CCtT),  then  P{gCC(.TT),q)  £ 9A}  = 0.)  Then 

C4.9)  p{a;  gCC(xT),a)  e 9A}  = 0 for  almost  all  ? (t7) 
values,  C5C*)f  measure). 

By  (.4.9),  and  the  convergence  C (x^)  C(x^)  w.p.l 

CSkorokhod  imbedding  used) , and  the  properties  of  A and 
9A, 

(4.10)  pCa:  gCc'^CxJ”)  ,a)  e A)  ^ p{a:  g(5(xT),a)  e A) 

for  almost  all  ? (xT)  values  (5(*),Xj  measure).  Now,  note  the 
following  (m(*),q,  are  as  in  Part  1,  and  the  tj  are  < «>)  : 


Em(5  (t 


h- 


. nx“  );  x“  nx“,  j < q)I  h 

11  1 1 - 


a^VeA} 


C4.ll) 


= EmCC^Ct.  nx^“);  x!?  fix^,  j < q)P  ^ 

J i J ± -*■ 


J 


The  l.h.s.  of  C4.U)  converges  to 


(4.12) 


Em(^ (t . n tT) , T n T. , j < q)P  (65  (T. ) e A}. 


(4.13)  P e A}  p(a;  g(?(T.),a)  e A), 

f^n  1 j. 

i 

(a  3^  measurable  function)  for  almost  all  ^ (x^)  values 

(?(*)f  measure).  Thus,  the  l.h.s.  of  (4.11)  also  converges 

to  (4.12)  with  the  r.h.s.  of  (4.13)  replacing  the  indicator 
in  (4.12).  Now,  the  arbitrariness  of  m(*)»  q»  t ^ , j < q? 
implies  (4.2)  for  A of  the  chosen  class.  Since  such  A 
generate  the  Borel  algebra  over  R , (4.2)  holds  as  stated.  Q.E.D. 


Theorem  4.2.  Under  (Al.l)  to  (Al.4),  ^(*)  and  X(*) 
(given  by  (1.1))  induce  the  same  measure  on  D [0,“). 

Proof.  Define  a jump  time  of  X(*)  to  be  a jump  time 
of  Q(*).  Then  the  interjump  times  for  X(*)  are  mutually 
independent,  and  exponentially  distributed  with  mean  value 
1/c . 

Furthermore,  the  conditional  distribution  of  the  value 
of  the  i^^  jump  of  X(*)  (given  X(s),  s < i^^  jump  time,  and 


I 


1 

i 

1 


( 


i 

i . 


1. 
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jump  time,  j < i)  is  the  same  as  for  C(*).  Between  jumps 
the  processes  evolve  as  diffusions.  Thus,  by  CAl.3),  the  induced 
measures  are  the  same  for  ?(•)  and  X(*).  In  particular,  the 
5 C* ) measure  does  not  depend  on  the  subsequence.  Q.E.D. 


5.  An  Alternative  Representation  for 


J^(-) 


and 


£^(•1 


The  representation  to  be  developed  in  this  section  is 
particularly  useful  when  we  treat  the  control  case,  for  it  will 
allow  us  to  prove  optimality  theorems  along  the  lines  of  those 
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h . . r 

an(I  J C* ) is  tight  on  D [0,«>)  . (The  integrations 
fti+l 

w are  always  over  the  interval  (t^,t^^,].  Only  the 

; 1 X+JL 

jumps  of  Q(.)  on  ^ role.  Define 

?^C0“)  = 5^(0),  etc.) 


Proof.  Define  s^  = min{t;  t > t^',  3^(t) 

V / 


- Q (t")  0), 


= 

1 


®i'^^i+l 


g(^^,a)Q^  (daxds) 


Then,  for  any  Borel  set  A, 


^h  , -hi 


(5.3a)  P{j^  + F.‘  e a1s“,  j < i,  < t^^^}  = 


r(Ci,A), 


(5.3b)  P(jJ  + j^)  |5y  j ; i,  sj  < } . 


Let  us  construct  a new  chain,  also  called  » as 

follows.  If  Q^(*)  does  not  jump  in  then  let 

evolve  from  using  the  Markov  law  p^ (•,•),  as 

before.  If  Q^(*)  does  jump  in  ' then  set 
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Ci+i  - = Cj{h,,..,j^h)  if  e 

The  law  of  the  new  chain  is  exactly  the  same  as  the  law  of  the 
former  chain. 

The  difference  ~ - (j^  + e^) 

X ^x+1  ^x'  {^ump  at  x+1}  '-^x  x 

is  due  to  the  method  of  discretizing  the  jumps,  and  {e^(*)} 
converges  weakly  to  the  zero  process,  as  h ->  0.  Define 
+ e^,  with  interpolation  E^{»).  Then  (5.1) 
holds,  and  E (•)  "^  zero  process  weakly,  as  h 0.  Tightness 
of  {J^C’)}  follows  from  the  properties  of  {Q^(*)}.  Q.E.D. 


Theorem  5.2.  Assume  the  conditions  of  Theorem  5.1,  and 

let^  g(*,*)  be  continuous.  Then  ( • ) f ( • ) / 

'^h  T 4]r+2r  * 

Q (•))  is  tight  on  D [0,°°)  . Let  h index  a convergent 

subsequence,  with  limit  5 ( • ) ,B  ( • ) , J ( • ) , 0 ,Q ( • ) . Then  Q(0 
is  a Poisson  process  of  the  type  of  (Al.2)  (with  correspondin 


Poisson  measure  Q(*,*)),  and  there  is  a standard  Wiener  process 


W ( • ) Cwe  may  possibly  have  to  augment  the  space,  by  adding  an 

/N 

independent  Wiener  process)  such  that  Q(*)  and  W(*)  are 


independent,  ? ( • ) ,B  ( • ) , J ( * ) are  non-anticipative  with  respect 
to  Q{*)  and  W(*)f  and 


(5.4)  5(t)  = X + f(5(s))ds  + B(t)  + J(t), 

J 0 


(5.5)  B(t) 


|-t 
■ 0 

=i: 


a (5  (s)  )dW(s) 


(5.6)  J(t)  = g(C(s  ) ,a) Q (daxds)  . 

0 J 


^The  continuity  has  and  (also  in  Theorems 6.1  and  8.2)  can  be 
removed  by  use  of  a more  "careful"  choice  of  the  discretization 
for  the  jump. 


1 


Proof.  The  tightness  follows  from  Theorems  3.1  and  5.1. 

A 

Also,  QC*)  is  a Poisson  process  of  the  asserted  type  since 
each  Q C* ) is.  The  representation  (5.4)  follows  from  the  weak 
convergence. 

A modification  of  the  proof  of  Theorem  6.3.2  in  [3]  yields 

A 

that  B t* ) is  a continuous  martingale  with  respect  to  {^(t)}. 


where  ^Ct)  is  the  smallest  a-algebra  which  measures 

A 

CCs),  J(s),  Q(s),  s < t.  The  quadratic  covariation  of  B(*)  is 


2a(C(s))ds.  Augment  the  probability  space  by  adding 

0 


an  independent  Wiener  process,  and  let  ^(t)  be  the  smallest 

A 

o-algebra  which  measures  ? (s)  , J(s),  Q (s)  ( (s)  , s < t.  By 

the  method  of  Chapter  1.4.4  and  Theorem  6.3.2  of  [3],  we  obtain 
a W(*)  (non-anticipatively)  from  ?(•)/  B(*)  such  that 
{(W(t),  _^(t)  } is  a Wiener  process  and  (5.5)  holds  for  all  t, 
w.p.l.  This,  together  with  the  fact  that  Q (•)  is  a Poisson 
process,  implies  that  {Q(t+s)  ~ Q(t),  W(t+s)  - W(t),  s > 0} 
is  independent  of  {Q(u),W(u),u  < t}  for  each  t.  Thus 

A 

(W(*),QC*))  has  independent  increments  (see  also  Section  9) 

» A 

and  since  W(*)  is  continuous  and  Q(*)  is  a pure  jump 
process,  by  Gikhman  and  Skorokhod  [5] , p.  271,  they  are  mutually 
independent. 

We  have  only  to  prove  the  representation  (5.6).  Define 


C(0")  = C(0)  , and 
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where  Cresp.  q^)  are  the  jumps  of  Q^(*)/  and  Q(*)f 

respw  and,  by  j e A,  we  mean  that  we  sum  over  only  the  j 

■4~ 

for  which  the  j jump  occurs  in  the  interval  (iA,iA+A]. 

h h 

By  weak  convergence,  we  have  q^  ->  qj,C  (iA)  -*•  C(iA) 
Cw.p.l),  and  using  the  fact  that 

P{jump  of  QC*)  at  iA  or  iA  + A}  = 0,  and  the  continuity  of 
gC*,*)  we  get  that 


B(A,h,i)  -»■  B(A,i)  w.p.l,  as  h 0.  Q.E.D. 

Remark  on  a useful  representation  for  Augment 

the  space  on  which  are  defined  by  adding  an  independent 

standard  valued  Wiener  process  In  Theorem  6.6.1  of 


[3]  (where 


g(*,*)  = 0),  it  was  shown  that  we  car  write  (for  some 


where  the  interpolation  of  converges  weakly  to  the  zero 


process  and  5 wJJ  (and  ejj)  depend  on  cjj.  " 'I' (tjj)  . 

Also,  the  process  W^(*)^  defined  by  the  interpolation  of 
h Ji 

~ I ' converges  weakly  to  a standard  R^  valued  Wiener 
i=0  " 

W(’),  and  (5.5)  holds  with  this  W(*).  We  can  do  the 

same  thing  here,  defining  6W^  exactly  as  it  was  defined  in 
[3],  Theorem  6.6.1,  and  Theorem  5.2  holds  if  W(*)  is  con- 
structed as  a limit  of  {W^(*)}.  We  then  can  write 


process. 


I I h g<Cn<i>)Q'’(da><ds)  + eJJ, 


for  some  {eJJ}  such  that  E^(*)  -►  zero  process  weakly,  as  h 


It  is  conceivable  that  the  coefficients  of  all  terms  in 
(2.2)  such  that  ^ ~ zero.  Then,  when  there 

is  a jump,  the  jump  could  be  strictly  zero.  If  these  probabilities 
are  small,  then  its’ not  important  - but,  if  not,  then  we  may  be 
able  to  save  some  computation  by  using  a slightly  different 
chain  {cjj}.  Define 

c(x)  = cp{a:  g(x,a)  0} 

F(x,A)  = p{a:  g(x,a)  e A - {0}}/p{a;  g(x,a)  0}. 

Assume  that  c(")  is  continuous.  Equation  (1.3)  can  be  written  as 

(5.8)  ^V{x)  + c(x)  [V(x+a)  - V(x)]f(x,da)  + k(x)  = 0. 

. 

Now,  proceeding  exactly  as  was  done  in  Section  2,  and 
defining 


At^(x)  = h^/[Qj^(x)  + c(x)h^] 

P^(x)  = 1 - exp  - c(x)At^(x), 

yields  (2.4),  where  all  other  terms  are  as  defined  there, 
except  that  (defined  as  was  defined,  but  using  T 

in  lieu  of  D replaces  there. 
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The  At  Cx)  are  larger  than  before  - since  we  have 
eliminated  the  zero  jumps.  With  the  new  definitions, 

Theorems  3.1  and  3.2  remain  valid.  Theorem  4.1  needs  to  be 
modified  as  follows.  Here,  let  a^,,..,  denote  the  times  of 
the  actual  Cnot  zero)  jumps  of  CC-),  and  let  be  the  smallest 

CT-algebra  which  measures  C (s) , s < a^,  a^,  j < i.  The  second 
line  of  (4.1)  remains  valid,  if  c is  replaced  by  c(C(t)), 
and  the  third  line  remains  valid.  The  right  side  of  the  first 
line  of  (4.1)  must  be  replaced  by 


(5.9) 


|-tH 
■ t 


c (5  (s)  )ds. 


where  |’C*)  is  a diffusion  process  df  = f(^dt  + o(C‘)dw  on 
[t,<»),  with  initial  condition  C (t)  , but  which  is  otherwise 
inde’^endent  of  5 (* ) . 

Equation  (4.2)  must  be  replaced  by  (what  is  expected) 


(5.10)  P^.{5tCTi)  - ?Ca~)  e A)  = r(C(a"),A) 


for  each  Bor el  A. 

Theorem  4.2  also  remains  valid  for  the  following 
reason.  Between  jumps,  both  X(*)  and  C(*)  behave  as 
diffusions,  and  the  conditional  distribution  of  the  jumps  is 
the  same,  where  we  now  define  a jump  to  be  an  actual  non-zero 
jump  (of  C(’)  or  of  X(*)).  Furthermore,  the  distribution  of 


the  interjump  times  is  the  same  for  both,  namely 


J. 


"1 


(5.U) 


> tlcCs),  s < C (a^)  = y} 

ft 

= E exp  - c(C(s)  )ds. 

^ 0 


= E exp  - c(C(s)  )ds. 

^ J 0 

which  is  also  the  corresponding  distribution  for  X(*). 

The  details  are  similar  to  those  of  the  foregoing 
proofs,  except  for  C5.9)  and  (5.11),  which  are  a little  more 
involved.  Also,  the  construction  of  C5.7)  can  be  carried 
out  for  the  new  chain,  and  Theorems  5.1  and  5.2  remains  valid. 


6.  The  Case  0 = OO 

We  now  consider  the  case  where  the  jump  rate  is  infinite, 
but  where  "most"  jumps  are  very  small. 

Assume 

CA6.1)  TT{A)  < " for  each  A which  is  disjoint  from  the 
origin.  (Then,  for  each  e > 0,  ttC*)  is  a finite  measure  on 


- {x;  |x|  < e}. 


(A6.2)  There  is  a real  K such  that  lg(x,a)(  <K|a| 


CA6.3) 


a I IT  Cda ) < ® 


alirCda)  -►0  as  e -*■  0. 


a <e 


I 
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other  sets  of  assumptions,  besides  A6.1-A6.3  can  be 
used.  For  example,  each  component  of  the  vector  Q(.)  can 
be  treated  separately  with  the  appropriate  assumptions  on 

g C* , • ) . 

For  each  e > 0,  let  g^(*,*)  denote  a function 

satisfying  (Al.l)  and  CA6.2)CK  independent  of  e) , and  which 

is  zero  when  laj  < e/2,  and  equal  to  g{*,-)  for  |a|  > e. 

^ r h 

If  rCx,R  ) = 00  for  some  x,  then  we  would  have  At"(x)  = 0, 

and  the  approximation  procedure  in  Section  2 has  to  be  modified. 

(This  makes  intuitive  sense,  for  the  interpolation  interval 

should  decrease  as  the  jump  rate  increases.)  Choose  some  sequence 

(to  be  further  restricted  below)  e^  ->  0,  as  h 0,  and  define 
/% 

r^(x,A)  = IT  (a;  g^(x,a)  e A).  With  difference  interval  h,  we 
discretize  (6.1)  in  lieu  of  the  current  form  of  (1.3). 

f (x,A)  = TfCa;  g (x,a)  e A) 

^h  ^h 

(6.1)  S^Vix)  + [ (V(x+a)  - V(x)]r  (x,da)  + k(x)  = 0,  x £ G. 

Define  T (x,j as  F (x,j,,...,j  ) was  defined, 

Ej^  j.  r X r 

^ r 

but  using  F_  in  lieu  of  F.  Define  c.  (x)  = ) F (x, j .,..., j^) , 

^ {j.}  ^h  ^ "" 

where  the  sum  is  over  all  (jj^f..»j^)  for  which  (j|»...fj^)  / 0. 
The  jumps  of  zero  value  are  to  be  deleted,  and  we  will  use 

At^(x)  = hV[Qj^(x)  + h^Cj^(x)] 

P^(x)  = [1  - exp  - Cj^(x)  At^(x)  ] or  [c^  (x)  At^  (x)  ] , 

analogously  to  (2.3). 


j 


-31- 


1. 

I. 


Define  p^Cx,y)  as  in  Section  2.  Then  we  get  the  discretized 

aJi 

form  C2,4),  but  with  Cx, 

h h ^ h 

new  At^Cx)  and  p”Cx)  replacing  r (x,  . . . , j^)  , c and  the 

former  At^Cx)  and  P^(x)  . 

Theorem  6.1.  Choose  0j^  such  that 

max  c.  (x)At"(x)  -*■  0,  max  Tr(a;  |a|  > E,/2)At"(x)  -»  0,  as  h ->  0. 

X “ X 

I 

Assume^  CaI . 1 ) ^ CAl . 4 ) and  (A6 . 1 ) (A6 . 3 ) . Then  (omitting 

the  assertions  concerning  and  6t^)  Theorems  3.1,  3.2, 

5.1  and  5.2  Cif  g(*,*)  is  continuous)  continue  to  hold;  in 

j 

Theorems  5.1,  5.2,  we  need  to  add  to  (the  previously  defined)  j 

h ® h i 

E (•)  a process  E (•),  which  also  tends  to  zero  weakly  as 

h 0 . Also,  the  remarks  after  Theorem  5.2,  concerning  ( • ) j 

j 

and  its  limit  W(*),  continue  to  ho Id I | 


Proof.  Tightness  of  {F^ (• ) » C* ) } follows  from 

Theorem  3,1.  The  value  of  e,  is  chosen  to  assure  that  E^(*) 

n 

h 6 

(see  Theorem  3.1)  converges  to  the  zero  process.  Let  denote 

J^(*)/  but  where  all  jumps  of  absolute  magnitude  < 6 are  deleted. 
Then  by  Theorem  3.1,  for  each  6 > 0,  {J  ' (*)}  is  tight.  By 
(A6.2)  - (A6.3)',  there  is  a real  such  that  for  each  T < «>, 

6 > 0, 


E max  |J^Ct)  - J^''^(t)|  < K.6T. 
t<T  " ^ 


] + ^ ^ \ 

Where  Al.2  is  modified  to  take  account  of  the  jump  properties 

i A 

’ of  Q(*)  assumed  in  this  Section. 


The  last  two  sentences  imply  that,  for  any  subsequence  of 

we  can  find  a further  subsequence  that  converges  weakly  in  D^[0,®) . 

Since  tightness  is  used  only  to  get  weak  convergence,  we  can 

assume  that  {J^C*)}  is  tight.  Theorems  3.1  and  3.2  (except  for 

the  assertions  concerning  follow  from  these  remarks. 

For  Theorems  5.1  and  5.2  (with  the  appropriate  generalize- 

tion  of  CAl.2))  define  e.  using  g (*,*)  in  lieu  of  g(*,*),  and 

^ ^h 

define  s^  = min{t:  t > t^,  |'3^(t)  - Q^(t  ) | > Cy^/2} . Define 


eh 

^i  = 


i+1  y. 

[g^  (q.a) 

^h  ^h  ^ 


g(Ci/a)]Q^(daxds) , 


h _ h 
n A • 
j=0  J 


^ h ® h. 

We  continue  to  define  using  g(*,*),  as  in  (5.2).  Both  E (•) 

— h 

and  £(•)-►  zero  process  as  h 0,  by  (A6.1)  to  (A6.3).  Modify 
"'h  *h 

in  (5.1)  by  adding  E . Then  Theorem  5.1  continues  to  hold. 

For  Theorem  5.2,  the  assertions  there  concerning 
A(A,*)  and  A^(A,*)  continue  to  hold  in  the  present  case,  and 
we  only  outline  the  convergence  of  B(A,h,i)  to  B(A,i).  By 
(A6.1)  to  (A6.3),  both 
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I IgU^(iA)  1 f E I |g(?(iA),q.) 
jeA  jeA 


kj 


kj  l5>S 


tend  to  zero  as  6-^0,  uniformly  in  h.  So  v;e  only  need  to 
show  convergence  when  | q j I k^l  restricted  to  be 

> 6,  for  6 arbitrarily  close  to  zero.  Choose  any  6 > 0 
such  that  7T(a:  la|  = 6)  = 0.  Then,  by  a weak  convergence 
argument  (using  Skorokhod  imbedding),  as  in  Theorem  5.2,  we  get 


I gC5^CiAl,qk 
jeA  ■' 

kjlxs 


I gUCiA),q.) 


kj  I 


w.p.l,  as  h 0.  Q.E.D. 

« 

Remark.  The  fact  that  Theorem  5.2  holds  implies  that 
C(*)  and  X(*)  have  the  same  distributions  under  (Al.3),  if 
g(*,*)  is  continuous.  Suppose  that  ?^(*)  is  constructed  by  the 

discretization  of  this  section,  but  without  assuming  continuity 
of  gC*,*)  in  the  second  argument.  Then  Theorem  4.2  still 
holds.  The  proof  is  more  complicated  and  is  omitted. 


7.  Convergence  of  the  Functionals  V (x)  to  R(x) 


The  chain  can  be  used  to  approximate  a large 

class  of  functionals  of  X(*),  and  of  solutions  to  equations 


such  as  (1.3). 


Theoram  7 . 1 (see  [ 3 ] , Theorem  6.4.1  and  [ 6 ] , 

Chapters  2 and  4).  Under  (Al.l)  ^ (Al.4)  (and  (A6.1)  - (A6.3), 
if  c = a>)  , if  F(*)  is  any  real  valued  bounded  and  measurable 
function  on  D^[0,oo)  which  is  a.e.  continuous  with  respect  to 
the  measure  induced  by  CC*)  ££  X(*)»  with  initial  condition  x, 

then 

E^FC|*^(-))  -^E^FC^C-)),  as  h -v  0. 


The  theorem  holds  if  uniform  integrability  of  { F (?"(•)) ) 
replaces  boundedness ^ and  the  convergence  is  uniform  on 
compact  X sets. 

To  treat  V^.Cx)  -»■  R(x)  , in  particular,  some  more 
conditions  are  needed,  since  we  need  to  know  that  our  func- 
tionals are  a.e.  continuous  (as  in  the  theorem)  and  that  the 
approximations  are  uniformly  integrable.  Define  t(*):  D^[0,«>) 
[0,«>]  by  t(x(*))  = inf{t;  x(t)  ^ G} . We  need  to  assume 
that 

(A7.1)  t(*)  is  continuous  w.p.l  (X(*)  measure, 

X(0)  = x) ; i.e.,  that,  w.p.l,  there  are  no  path  tanqencies  at 
the  point  of  contact  of  X(*)  with  9G. 


We  also  need  that 


CA7.2)  3.S  uniformly  bounded  in  h (to  get 

uniform  integrability) . This  is  implied  by 


inf  P {escape  time  of  X(*)  from  G < T}  > 0 
xeG  ^ 


for  some  T < <».  See  [3],  Chapter  6.4. 


Now,  under  conditions  (Al.l)  to  (A1.4)  (and  (A6.1)  to 
(A6.3),  and  continuity"^ of  gC*,*),  if  c = ■»)  , and  (A7.1)  - 
(A7.2),  and  = x,  we  have  that  CC*)  (which  has  the 

law  of  XC*),  X(0)  = x)  as  h -»■  and 


())Cr  (P}^))  ^ 4>(aT)) 

p. 


k(5“(s))ds  ->  k(C(s))ds 
0 J 0 


in  distribution  (and  the  mean  values  also  converge;  i.e., 
V^(x]  •+  R(x)),  as  h 0. 


8.  An  Alternative  Interpolation  C“(*) 


In  [ 3 ] and  in  Section  2,  the  interpolation  5 (•)  is 

h h 

constant  on  time  intervals  {At?}.  Given  = x,  the  inter- 
polation  interval  [t?,t?.,)  is  known,  and  C^(*)  is  not  a 


Theorem  6.1  assumed  the  continuity,  but  Theorem  4.2  still  holds 
when  c = «>,  although  the  proof  was  not  given.  Under  (the 
extended)  Theorem  4.2,  the  continuity  of  g(*,*)  can  be  dropped, 
in  favor  of  (Al.l). 
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Markov  process.  For  some  purposes,  it  would  be  convenient  if  C (*) 
were  Markovian.  In  this  -section,  a right  continuous  Markov  interpola- 
tion, with  random  (exponentially  distributed)  interpolation  intervals, 
will  be  developed.  It  has  the  additional  advantage  that  sup  At^(x) 

X 

need  not  be  finite,  and  the  condition  (Al.4)  can  be  dropped.  We  assume 
c < “,  although  there  is  an  analogous  development  for  the  case  of 
Section  6.  Unless  otherwise  specified,  symbols  retain  their  earlier 
definitions.  First,  the  simple  case  g(*,*)  = 0 will  be  treated. 


8.1.  Case  I g(*,*)  = Q{*)  = 0 (the  case  of  [3], 

Chapter  6).  Let  At^(x)  = h^/Qj^(x).  Define  C^(*)  (we  use 
the  same  symbol  for  the  new  interpolation)  to  be  a Markov  jump 
process  as  follows.  Let  ^^(t)  be  the  smallest  a-algebra  which 
measures  C^(s),  s < t,  set  Tq  = 0 and,  for  i > 1,  let 
and  denote  the  i^^  jump  time  and  interjump  time, 

resp. , of  C^C*).  Assume  that 


(8.1) 


P{?^C*)  changes  in  (t,t+A]  [^^(t)  ,C^(t)  = x}  = 1 - exp  - (A/At^(x)), 
P{?^(T^)  - = y|  5^(Ti_i)  = x}  = p^(x,y),  all  x,y. 


By  (8.1), 


(8.2) 


E[t^^1-tJu^(s),  s < tJ]  = At^(C^(T^)) 


Owing  to  (8.2),  and  to  the  law  of  C^^(t^)  - the  solution 
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to  C2.4]  with.  (P^C*)  = c = = 0 here)  still  has  the  repre- 
sentation C3.5)/  where  is  the  escape  time  of  the  new  C^(*) 

from  G.  The  feet  that  V^(x)  still  has  the  representation 
C3.5)  is  critical  for  our  method.  This,  together  with  the 
(to  be  proved  under  (Al.3))  fact  that  the  sequence  of  new  "random" 
interpolations  converges  weakly  to  (1.1),  implies  that  the  results 
of  Section  7 still  hold  for  our  new  interpolation.  This  justifies 
the  use  of  this  new  interpolation  also.  Thus:  cost  functional  for 

chain  = cost  functional  for  interpolation,  which,  in  turn,  converges 
to  R(x).  The  same  is  true  for  the  case  g,  Q ^ 0,  discussed 
below.  The  actual  computations  of  the  approximations  to  functionals 
of  X(*),  or  to  solutions  of  (1.3)  will  still  be  done  using  the  law 
of  the  chain.  The  interpolation  is  used  only  for  the  theoretical 
arguments  in  the  convergence  proofs. 

Define  ^jj^x)  = a(x)o'(x)  + hf  (x)  = ^j^(x)  - f (x)  f ' (x)  (At^  (x)  ) ^ 

the  nature  of  the  interpolation  C (*),  the  (mean)  term  does  not 
appear  in  the  expression  for  the  quadratic  covariation. 

Theorem  8.1.  Assume  (Al.l),  (A1.2),  and  fix  = 

C^CO)  = X.  Let  gC'»‘)  = 0.  Then  we  can  write 

(8.3)  C^Ct)  = X + F*^(t)  + B^(t), 


where 


r 


-38- 


F^Ct) 


B^Ct) 


= f 


f (5"‘Cs))ds 


= f^[d5^(s)  - fU^(s))ds]  = 

J 0 


’dB^(s), 


and  B (.* ) is  a martingale  with  quadratic  covariation 


t_ 


is)  )ds. 

0 ” 

The  sequence  F^C*)f  B^C*)}  is  tight  on  D^^[0,“), 

and  for  any  weakly  convergent  subsequence,  with  limit  denoted 
by  C C* ) f FC*)»  B(*)f  there  is  a Wiener  process  W(»)  such 
that  C (-• ) / B(*)  are  non-anticipative  with  respect  to  W(>) 
and 


(8.4) 


^Ct)  = X + F(t)  + B(t)  = X + 


f (5  (s)  )ds 


+ I aCC  (s)  )dW(s)  . 
0 


There  is  a martingale  process  W^C*)  with  quadratic 
covariation  It  such  that 


(8.5)  B^Ctl  = rat?^(t))dW^(s)  + E^(t)  + [^[a.  (C*^(s))  - a(?^(s))]dw^(s) , 

Jo  . Jo  " 


vrhere  the  last  two  processes  in  (8.5)  tend  to  the  zero  process 
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weakXy,  as  h 0,  and  (7j^Cx)a^(x)  = — 

h,  -+•  0,  The  sequence  (W^C-))  is  tight.  Also,  any  weak  limit, 
WC*),  is  a Wiener  process  with  the  properties  of  the  W(*)  above. 
Under  CAl.3),  the  limit  ?C*)  is  the  unique  solution  to  (1.1) 

(in  the  sense  of  distributions) . 

Proof.  The  proof  is  close  to  that  given  for  the 
original  interpolation  in  [3],  Chapter  6.3  and  6.6,  and  only 
an  outline  will  be  given. 

Part  1.  Equation  (8.3)  holds  by  the  definitions. 

Fix  h.  Note  that 


E[?^Ct+A)  - 5^Ct)|^^Ct),C^(t)  = X]  = Cf(x)At^(x))-^+  o(A) 

At^(x) 


(=  o(A)  + conditional  (on  a jump)  average  change  in  C^(*)  over 

the  interval  (t,t+A]  times  conditional  probability  of  jump, 

which  is  — rp — + o(A))  from  which  the  martingale  property  of 
At"Cx) 

B^C*)  follows.  The  quadratic  covariation  formula  follows  from 


E[(B^(t+A)  - B^(t))  (B^(t+A)  - B^(t))'|^^(t),5^(t)  = x] 


= E[C?^(t+A)  - C^(t))  (C^(t+A)  - C^(t))'|  ^^(t),C^(t)  = x]  + o(A) 


= r.  (x)At*^(x)  C— r ) +o(A). 

At  (x) 
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Part  2.  The  tightness  of  F^(*)  is  obvious,  together 
with  the  fact  that  any  weak  limit  must  be  continuous  w.p.l. 

The  proof  for  closely  follows  that  of  [ 3 ], 

Theorem  6.3.1.  In  particular,  if  there  is  a real  K such 

that,  for  each  T > S, 

C8.6)  EjB^CT)  - B^CS)!"*  < K(T-S)^  + 0(h), 

then,  using  the  martingale  property  of  B^(*),  we  get  both 
tightness  and  continuity  of  all  limits,  as  in  [3],  Chapter  6. 

For  notational  convenience,  suppose  that  b"(»)  is  scalar  valued. 
We  will  evaluate 


E 


rT 
• S 


n dB^(s.) 

i=l 


by  evaluating  the  four  quantities  (a)  - (d) . The  l.h.s.  of  (8.6) 
is  bounded  above  by  a constant  times  the  sum  of  (a)  - (d) . Note 
that  B^(*)  has  an  absolutely  continuous  component,  and  a pure 
jump  component. 


(a)  E 


(b)  E 


(c)  E 


dB^(t) 


t 4 . . 

n dB^'(s . ) 

S i=2  ^ 


[dB^(tl]^  n dB^(s.) 

•'S  J i=3  ^ 


IdB^  (t)]^ 


dB^(s) 
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1. 


Cd)  E I IdB^Ct)]^. 


(.d)  is  clearly  bounded  above  by  some  Kh**  times  the 
average  number  of  jumps  in  [S,T] , which  is  less  than  (T-S)K/h^, 
for  some  real  K.  The  quantity  (a)  is  zero,  by  the  orthogonality 
of  the  increments  of  The  quantity  (b)  equals,  for  some 

real  K, 


^ E^CC^(t))dt 


t 2 , 

n dB^(s.  ) 
S i=l  ^ 


< K(T-S)  E 


ridB^(s.) 

i=l 


2 2 
< K^(T-S)  . 


Quantity  Cc)  is  bounded  above  by 


(dB^(t)  )^[  (dB^(t)  + ( 


'dB^(s))^], 


which  we  can  bound  by  using  the  results  for  (b)  and  (d) . 

Putting  all  the  estimates  together  yields  (8.6)  for 
some  real  K. 

Thus,  B^(*)}  is  tight.  Let  h index  a 

convergent  subsequence  with  limit  ?(•)»  B(*).  By  the  method 
of  Theorem  6.3.2  of  [3],  it  can  be  shown  that  B(*)  is  a 


L 
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h 


continuous  martingale  with  quadratic  covariation 


•t 

2a(5 (s) )ds, 

•'  0 


and  that  (8.4)  holds  for  a Wiener  process  with  the  asserted 
properties. 


Part  3.  The  proof  of  the  assertion  concerning  the 
representation  (8.5)  is  close  to  that  given  in  [3], 

i- 

Chapter  6.6,  for  a similar  result.  Let  (•)  denote  a 

standard  R valued  Wiener  process,  which  is  independent  of 

C^(*).  Choose  measurable  diagonal  orthonormal 

P.  (•)  matrices  such  that  Y,  (x)  = P.  (x) (x) P ' (x) , where 
h n h h h 

Oj^(x)  = Pj^(x)Dj^(x)  ->  a(x)  as  h ^ 0.  Define  ^^(t)  = 
Ij^(5^(t)),  P^(t)  = P^(C^(t))  and  (t)  = D^(?^(t)). 

Let  (d^ (t) , . . . ,d^ (t) ) denote  the  diagonal  elements 
of  Dj^(t).  Choose  a e (0,1).  Define  the  diagonal  matrices 
Dj^(t)  ,Dj^  (t),Dj^(t),  resp.  , as  the  matrices  with  i diagonal 
elements  (t) j (t) ^ and 

^i^<^^^{d.  (t)>h«}' 

Define  W^(t)  by  W^(0)  = 0 and 


dW^(t)  = Dj^(t)P^(t)dB^(t)  + (I-Dj(t)I^(t))di);^(t). 

Then  (8.5)  follows  from^ 

dB^Ct)  = Pj^(t)D^(t)dW^(t)  + dE^(t)  = 0j^(C^(t))dW^(t)  + dE^(t), 


dE^(t)  is  defined  analogously  to  the 


) 

1 


above  (6.6.6)  in  [3]. 
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where  EC*)  is  a process  which  tends  weakly  to  the  zero 
process  as  h ->•  0.  The  assertions  concerning  the  tightness 
of  tw»(.n,  the  non-anticipative  property  of  W(*),  and  the 


representation 


(C(s))dW(s)  are  all  similar  to  the  proofs 


of  the  related  assertions  in  [3],  Chapter  6.6,  and  are 
omitted.  Q.E.D. 

In  [3],  Chapters  8 and  9,  to  prove  optimality  of  the 
limit  of  the  costs  for  the  discretized  problems,  it  was 
frequently  necessary  to  "discretize"  an  arbitrary  control  for 
the  continuous  process  X(*),  and  then  to  apply  this  to  the 
discrete  model.  The  W^{*)  obtained  in  this  Section  can  be 
used  instead  of  the  (•)  of  those  theorems  (e.g. , (3] 
Theorem  8.2.4,  etc.) 


8.2.  The  General  Case  (g,  Q ^ 0) 


Of  the  several  ways  in  which  a Markov  C (*)  can  be 

defined,  we  will  develop  one  that  is  particularly  easy  to 

relate  to  formulas  such  as  (2.4),  (2.5)  and  (3.5).  Pure 

li  li  h 

jump  right  continuous  processes  A (*),J  (•)  and  C (•) 
will  be  defined  such  that 


C8.71 


5^(t)  - C^(s)  = dA*’(u)  + f^dJ^(u) 

) S J s 

Define  ^^(x)  = h^/Qj^(x).  Note  that  (x)  ^ 
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defined  as  At^Cx)  was  in  Section  8.1,  but  not  as  At^{x) 
was  in  Sections  2^7.  The  processes  are  defined  by  the 
following  relations.  Let  SCt)  denote  the  set  {?^(s) ,A^(s) , 

J^ts),  s < t}.  All  o(*)  are  uniform  in  u),t,  but  not  necessarily 

in  h. 

C8.8a)  P{2  or  more  jumps  of{A*^(*),  in  (t , t+A]  | S (t)  } = o(A), 

(8.8b)  P{A^(*)  jumps  in  (t,t+A)  | S(t),?^(t)  = x}  = A/^^(x)  + o(A), 

(8.8c)  P{J^(*)  jumps  in  (t , t+A] | S (t) } = cA  + o(A). 

Let  and  denote  the  i^^  jump  times  of  A^(*)  and 

(• ) , resp.  Then  let 

(8.8d)  P{A^(Tj^)  - A^Ct")  = y|  S(tT)  = x}  * p^(x,y), 

(8.8e)  P(J^Ca^)  - J^(a“)  = (j]^h, . . . , j^h)  | S (o')  ,o  (a")  = x)  = 

Cjj^f-fjj.  corresp.  to  j|,...,j^) 

(see  Section  2).  Note  that  (8.8e)  allows  jumps  of  J^(*)  of 
zero  magnitude,  as  in  Sections  2 to  4. 
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Suppose  that  C^(*)  jumps  at  t,  with 
C^Ct)  = ?^Ct^l  = X.  Then  C^C*)  is  constant  until  the  next 
jrmp.  The  interval  until  the  next  jump  is  the  smallest  of  two 
random  variables,  one  exponentially  distributed  with  mean  1/c, 
the  second  exponentially  distributed  with  mean  ^^(x).  Thus, 

S(t)  with  C^(t)  = x,  the  probability  that  the  next  jump 

t is  that  of  rather  than  that  of  A^(*)  is 

/ 

Cx)  c 

1+At^(x)c 
^ h 

which  is  precisely  P (x)  (see  (2.3a)).  Thus,  the  probability 

that  J^(*)  jump  next  is  just  the  probability  that  the 
ii  h li 

change  in  S^i+l  ” (^iven  = x}  and  using  (2.3a))  is  due  to 

the  "jump  component"  of  the  process 

The  average  (conditioned  on  S(t),C^(t)  = x)  time  until 
the  next  jump  after  t of  C (•)  is 


given 

after 

. (8.9) 


(8.10) 


At*^(x) 
c^^  (x)  +1 


which  is  precisely  the  At^(x)  of  Section  2.  Let  {u^}  denote 
the  jump  times  of  C^(*)«  Then  the  distribution  of 


^"(Uj^^j^)  - 5^(Uj^),  given  c”(Uj^)  = x,  is  the  same  as  that  of 
w w w /expression  for  the 

given  = X.  This,  together  with  the' conditional 

average  waiting  time  (8.10),  implies  that  (3.5)  remains  the 

solution  to  (2.4),  where  ^ (•)  is  the  process  just  constructed, 

and  p. 

h 


its'  escape  time  from  Gj^. 


-46- 


Thus,  the  new  interpolation  makes  sense,  for  our  purposes. 

It  is  clearly  a Markov  jump  process,  and  can  be  used  to  study 
the  limit  of  V^Cx),  and  of  other  functionals  of  the  chain, 
provided  that  X(*)  in  distribution,  as  h ->•  0,  and 

(Al.3)  holds. 

Using  the  definition  of  B^{*)  and  F^(-)  of 
Section  8.1,  and  letting  ^^(0)  = x. 


C8.ll) 


^ (t)  = X + 


f (?^(s)  )ds  + B^(t)  + J^(t) 


Let  Q C* f • ) denote  a Poisson  measure  (with  corresponding 
^h 

Poisson  process  U (•))  satisfying  (Al.2).  Then  we  can  assume 


that  J (•)  has  the  representation 


C8.12) 


J*^(t) 


gC5^(s  ) ,a) (daxds)  + E^(t), 


where  E^{*)  •>  zero  process  weakly,  as  h ->■  0,  and  C^C*)  is 

''h  ''h 

non-anticipative  with  respect  to  Q (•)•  The  E {•)  repre- 
sents only  the  indefinite  sums  of  the  difference  between  the 
jumps  in  the  integral  in  (8.12)  and  the  points  on  the  grid 

G.  to  which  our  convention  assigns  these  points;  e.g.,  the 
h 

assigned  point  is  ( j [h, . . . , j^h) , if  the  jump  is  in  the  box 

b^Cjj^, . . . , j^)  . We  can  assume  the  form  (8.12),  in  the  sense 

h.  ^ h h 

that  there  are  processes  ^ (•),E  (* ) » A (• ) satisfying  the 
properties  below  (8.12)  and  such  that  (8.3)  continues  to  hold 
and  where  5 (•)  is  defined  by 
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C8.13) 


5^Ct)  ==  X + A^{t)  + f g(.C^(s“),a)Q^((3axds)  + E^(t) , 

h 


A^(t)  = F^Ct)  + B^(t), 


and  the  C?^C* ) »F^C* ) /B^(« ) /J^(* ) + E^C*))  in  (8.13)  induce  the 
same  measure  on  d'^^[0,«>)  as  does  the  set  in  (8.11)  (with  E^(*) 
deleted).  The  only  purpose  of  ( • , • ) is  to  generate  the  jumps 
of  J (♦),  according  to  the  appropriate  distribution.  Define 
( • ) as  in  Section  8.1. 


Theorem  8.2.  Assume  (Al.l)  - (A1.2).  Define  c"(*)» 

(• ) (• ) f J^(* ) (and  E^(*),Q^(.)/  where  appropriate)  by  the 
construction  leading  to  either  (8.11)  or  (8.13).  The  assertions 
of  Theorem  8.1  concerning  B*^(*)  and  W^(*)  continue  to  hold. 

The  processes  above  are  all  tight,  and  E^(*)->  zero  process, 

A 

Q (• ) Poisson  process  Q (• ) , weakly  as  h 0.  Let  h index 
a convergent  subsequence  of  the  other  processes ^ with  limit 
? (• ) /F(« ) ,B(* ) ,J(* ) . There  exists  a Wiener  process  W(*)  (which 
can  be  the  limit  of  W^(*))  such  that 


?(t)  = X + f(C(s))ds  + B(t)  + J(t), 


where 


J 0 


(C(s)  )dW(s)  , 


and  5 C*  1 /B  (• ) , J C* ) Cand  Q(*),  where  appropriate)  are  non- 

anticipative  with  respect  to  W(*).  Indeed,  Q(*)  and  W(*) 
are  independent. 

The  process  J(*)  has  the  same  properties  as  the 
JC*)  in  Section  4 . In  the  case  of  the  construction  (8.13) 
and  if  g(-,*)  is  continuous. 


J(t)  = J J g(C(s  ) ,a)  Q (daxds)  , all  t,  w.p.l. 


Under  (Al.3),  the  limit  ?(•),  under  either  construction 


induces  the  same  measure  on  D [0,<»)  as  does  the  solution 


of  (1.1). 


The  proof  is  a combination  of  those  of  Theorems  4.1, 

4.2,  5.2  and  8.1  and  is  omitted. 

Under  (Al.3)  and  (Al.4),  the  processes  of  this  section 
and  that  of  Section  3 are  asymptotically  weakly  equivalent  to 
(1.1).  The  process  of  this  section  is  clearly  Markov.  The 
results  of  Section  7 hold  without  (Al.4).  The  process  in  (3.5) 
can  be  replaced  by  that  of  (8.11)  or  (8.13).  Equation  (3.5) 
holds  for  the  interpolation  of  Section  3 even  if  (Al.4)  does  not 
hold,  but  in  that  case,  the  weak  limits  of  those  interpolations 
are  not  necessarily  equivalent  to  (1.1). 
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9.  Optimal  Stoppping  Problems  and  Auxiliary  Results 

First,  we  prove  a result  which  is  useful  to  show  that 
a control  or  stopping  time  is  suitably  non-anticipative.  Let 
3 C* ) »QC* ) »w(* ) and  Y be  processes  with  paths  in  some 

p 

D [0,“)/  and  a random  variabler such  that  QC*)  is  a Poisson 
process  with  the  properties  of  (Al.l),  w(*)  is  a Wiener  process 
and,  for  each  integer  b and  bounded  continuous 

each  t > s (a  given  number)  and  real  numbers  tj^,...,tj^  < t,  let 
C9.1) 


f (9.3) 

Theorem  9.1.  Under  the  above  assumptions  with  s - 0, 

Q(*)/W(*)  have  independent  increments.  In  general 

{Q(s+u)  - Q(s),  w(s+u)  - w(u)  , u > 0}  is  independent  of  ■ 

(Q(u)  ,w(u)  , p (u)  , u < s,Y}. 

j 

Proof.  Since  the  first  assertion  follows  from  the 

second,  only  the  second  will  be  proved.  Let  Oq  = s and 

i 

t i 

■ I 

L j 


Ef^(Y)f2(Q(t^),w{t^),p(t^),  i < b)  (w(t+u)  - w(t))  =0,  u > 0, 

Ef^(Y)f2(Q(t^)  ,w(t^)  ,p(t^) , i < b)  A(t,u)  = 0,  u > 0, 

A(t,u)  = (w(t+u)  - w(t) ) (w(ttu)  - w(t)  )^  - ul 

{Q(s+u)  - Q(s),  u > 0}  is  independent  of 
(Q(u)  ,w(u)  ,p(u) , u < s,Y}. 
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{Oj,  j > 1}  denote  the  jump  times  of  Q(*)  on  (s,T] , where 
T is  an  arbitrary  number  > s.  Let  A(*)  and  n(*)  denote 
vector  valued  bounded  continuous  functions  on  [0,T],  and  let 
V denote  a vector.  Define 

A(t)  = exp  i[v'Y  + [ X»(u)dW(u)  + f ' (u)  dQ  (u)  ] . 

Jo  Jo 

By  the  assumption  (9.1)  - (9.2),  w(-)  is  a martingale  and  a 

Wiener  process  on  ts,T]  with  respect  to  {.^(t),  t > s}, 

where  ^(t)  = ^(Y,w(u)  ,Q  (u)  ,p  (u)  , u < t)  . Also,  the  a. 

J 

are  stopping  times  with  respect  to  { ^(t) , t > s}. 

We  have 


A(t)  = A(s)  + I [A(o;:)  - A(a.  .)]  + J (A(a.) 

j>l  ^ j>l  ] 

a.<t  a.<t 

D- 


A(Oj)  ] . 


By  Ito * s Lemma,  the  first  sum  is  (A(u  ) can  be  replaced  by 
A(u)  here) 


A(u  ) [iA  ' (u)dw(u) 

1 


a.<t 


U(u)  1^ 
2 


du]  . 


The  second  sxim  is 


where  X has  the  distribution  of  0(0^)  - QCOj),  the  jump 


in  Q(‘).  The  unique  solution  to  (9.4)  is 


C9.5)  ECt)  = E(s)exp[ 


U(u)  1^ 


i:' 


du  + I c ( (u)  - 1)  du]  . 
s 


The  result  follows  from  the  product  form  of  the  characteristic 
functional  in  (9.5),  since  the  exponential  term  is  the 
characteristic  functional  of  {w(s+u)  - w(s),  Q(s+u)  - Q(s), 
0<u<t-s}.  Q.E.D. 


Applications  of  Theorem  9.1.  Let  Q(*)/W(*)  denote 
the  limits  in  the  past  sections.  Set  s = 0,  and  drop  Y,  set 
Q(*)  =Q(*)f  W(*)  =w(*)f  CC*)  = P(*).  Equations  (9.1)  - (9.3),  and  the 


I 

i 

1 


assumptions 
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a,bove  them/  follow  by  the  type  of  weak  convergence  arauments 
used  in  the  proof  of  13],  Theorems  6.3.2  and  6.6.1.  In 
Theorem  6.3.2,  the  5 Cs^l  (with  or  without  the  superscript  h) 
are  to  be  replaced  by  ? (s^^)  ,Q  (s^)  ,W(s^)  (with  or  without  the 
superscript  h)  • 

Theorem  9.1  is  the  analog  for  our  case  of  [3], 

/are 

Theorem  8.2.1  and  Corollary  8.2.1,  which' used  to  show  that 
a stopping  time  p which  is  a limit  of  where  is 

an  optimal  stopping  time  for  an  optimal  stopping  problem  on 
the  chain,  is  non-anticipative  with  respect  to  W(*).  Here,  we 
would  let  Y = I{p<s}* 

Theorem  9.1  is  the  analog  - for  our  jump  case-of  [3] 
Theorem  8.2.1  and  Corollary  8.2.1.  These  theorems  were  used  to 
show  that  certain  limits  of  optimal  controls  or  stopping  times 
for  optimal  control  or  stopping  problems  on  controlled  forms 
of  actually  converged  to  non-anticipative  times  and 

controls  for  the  limiting  controlled  process.  The  same  thing 
is  done  here.  For  example,  in  the  context  of  [3] , Corollary 
8.2.1,  we  would  set  Y = i{p<g}/  where  p is  the  limit  of  the 
"approximate"  stopping  times,  and  show  (9.1)  - (9.3)  by  weak 
convergence  arguments  as  in  [3] . 

The  technique  in  [3],  for  proving  optimality  of  a 

/the 

limiting  control  or  stopping  time,  involved  comparing' cost 
under  the  optimal  policy  for  each  controlled  chain  to 

the  cost  under  a "discretized"  form  of  a policy  for  the 
controlled  diffusion.  The  same  thing  is  done  here.  We  only 


m 
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illustrate  the  analog  of  the  technique  of  [3],  Theorem  8.2.4, 
to  which  the  reader  is  referred.  The  remarks  below  will 
suggest  the  necessary  alterations  to  the  other  policy 
discretization  techniques  of  [3],  Chapters  8 and  9. 

Let  T < T denote  a stopping  time  for  (1.1)  which  is  a func- 
tional of  w(’)/Q{*).  Let  0 < 6 <<  A < T with  A/6  = q, 
an  integer.  Let  denote  the  smallest  a-algebra 

which  measures  {w(i6 ) ,Q (i6 ) , i6  < nA}.  Define 


p^C6,A)  = E^[x|  ^^^(w,Q)],  i > If 


and 


T C6,A) 


T/A 

ill Pi (6 , A) <iA, p j (6,A)>j6,j<i}‘ 


If  t(6,A),  or  any  other  stopping  time,  is  not  defined  at  some  u), 
set  it  equal  to  T there.  Let  the  decision  sets  for  x(6,A) 
be  A^C6,A)  e ) (i.e.,  x(6,A)  = i if 

(w(j6)  ,Q(j6)  , j6  < iA}  e A^(6,A)). 

e 

For  each  e > 0,  there  are  sets  A.  (6, A)  and  A .(6,A), 
which  are  open  and  closed,  resp. , and  which  satisfy: 


A.  (6, A)  DA.(6,A)  D A^  .(6, A), 

X X C f X 

y p{  Cw(j6)  ,Q(j6)  , j6  < iA)  e A^(6,A)  - A^  .(6, A)}  < e, 

X ~ ^ G ^ X • 


For  each  i,  there  is  an  open  set,  A?(6,A),  containing 
A .(6,A),  and  whose  closure  is  in  A^(6,A),  and  such  that 


P{  (w(j(S)  ,Q(j6)  , j6  < iA)  e 3A^(6,A)}  = 0. 


These  sets  {A^(5,A)}  are  used  to  get  the  approximate  stopping 
times,  just  as  the  b”  were  in  the  reference.  Define 


= {v  e 


j^iq(r+r»).  ^ ^ a?(6,A),  v i6  aJ(6,A) 


X R 


(i-j)q(r+r') 


j < i). 


Define 


T^(6,A)  = iA  if  {w( j6) ,Q ( j6) , j6  < iA}  e , 
= T if  not  otherwise  defined. 


J 

i 

: 1 


J 


i 

1 


The  t^(6,A)  with  (W^ ( j 6 ) ,Q^ ( j 6) ) substituted  for  (w ( j 6 ) ,Q ( j 6) ) 

is  the  appropriate  discretized  comparison  stopping  time  for 

(C  }•  The  rest  of  the  development  is  as  in  the  reference, 

■here 

except  that  e 0 ' replaces  n -»•  «>  there. 


I 

I 
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